In Software Defined Network (SDN) development environment, Agile or DevOps methodology and Continuous Integration (CI) are usually required. Agile software development relates to an approach for software development under which requirements and solutions evolve through the collaboration of cross-functional teams. Agile development promotes adaptive planning, evolutionary development, early delivery, and continuous improvement, and facilitates rapid and flexible response to change. Hence, these approaches are becoming more and more popular, and have been adopted by SDN software and network application development companies as a means of improving the quality of software, and reducing the amount of time it takes to deliver the software.
SDN Software-defined networking (SDN) is an umbrella term encompassing several kinds of network technology aimed at making the network as agile and flexible as the virtualized server and storage infrastructure of the modern data center. Due to the programmability of SDN, the risk of bugs and challenges for testing has been increased. As a result, the quality of SDN control software and network services actually determine the reliability of SDN.
Currently, testing is performed using a Model Based Testing approach. Model-based testing is an application of model-based design for designing and executing artifacts to perform software testing or system testing. Models can be used to represent the desired behavior of a system under test, or to represent testing strategies and a test environment. Because test suites are derived from models and not from source code, model-based testing is a form of black-box testing. The current Model-Based Testing based on the system description for application under test as finite state machines to generate test cases depending on the complexity of system under test, the corresponding model still could be very complicated and hard to use in real word.
Software correctness verification is a complicated subject. From a pure correctness verification perspective, people look for program invariants, and use Predicate Abstraction on the state space of program variables, Model Check etc. to verify its correctness. Program invariants are conditions that can be relied upon to be true during execution of a program, or during some portion of it. A program invariant is a logical assertion that is held to always be true during a certain phase of execution. For example, a loop invariant is a condition that is true at the beginning and end of every execution of a loop. The invariants are usually hard to find and for a middle-scale program usually has a large variable/predicate state space. With these difficulties, automated correctness verification is not used outside of a laboratory environment. From black-box testing perspective, methods to automatically verify service correctness for the SDN software are not available.
Currently test cases are usually managed by testers. Each person has his/her own test cases so that later on, the regression tests are sometimes hard to perform because there are no complete test scripts for a given release delivery. Often testers have to rewrite regression scripts to test same things which previously have been tested by others.
Currently tests typically use simulated data to do load/stress tests. This is because of lack of real data for systems. Many real world scenarios may not be tested so that the problems have to be faced after production deployment.
There is a need to provide methods for testing SDN software that avoids the disadvantages inherent in the model-based testing approach. There is a need to provide methods for testing SDN software that do not rely on program invariants to verify the correctness of the SDN software. There is a need to provide methods for testing SDN software that can be made available to a variety of users. There is a need to provide methods for testing SDN software that use real-world data to identify problems that have to be faced after production development.